On the dynamic programming approach for the 
3D Navier-Stokes equations 

Luigi Manca * 
February 2, 2008 

Abstract 

The dynamic programming approach for the control of a 3-D flow gov- 
erned by the stochastic Navier-Stokes equations for incompressible fluid 
in a bounded domain is studied. By a compactness argument, existence of 
solutions for the associated Hamilton-Jacobi-Bellman equation is proved. 
Finally, existence of an optimal control through the feedback formula and 
of an optimal state is discussed. 



Resume 

Nous etudions la programmation dynamique du controle d'un flux 
tridimensionnel gouverne par les equations stochastiques de Navier-Stokes 
qui concernent un fluide incompressible dans un domaine borne. Nous 
demontrons I'existence de solutions pour I'equation associee de Hamilton- 
Jacobi-Bellman par un argument de compacticite. Enfin nous examinons 
I'existence d'un controle optimal et d'un etat optimal au moyen de la 
formule de feedback. 
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1 Introduction 

In this article we study the dynamic programming approach for the control 
of a three dimensional turbulent flow governed by the stochastic Navier-Stokes 
equations for incompressible fluids. The unknows are the velocity field U{^, t) — 
(C/i(^,i),C/2(C,t),C/3(^,t)) and the pressure t), where t S [0, T] and ^ S -D, 
with Z3 C M"^ open and bounded. U{S,,t), p(£,,t) satisfy the equation 

+ {U -Vp + Vp = i^AU + z + rj inD 
divC/ = 0, in D 
f/ = in dD 
U\t=o^ Up in D. 
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where the control z ~ z(^, t) is a bounded random variable, v is the kinematic 
viscosity and 77 is a random variable of white noise type. Since v does not play 
a particular role, with no loss of generality we can assume that v = \. We look 
for a solution with values in the Hilbert space H of the square integrable and 
divergence free functions / : D ^ R'^. 

We consider a cost functional of the form 



J(z) 



($(c/(c,<)) + \\z{i,t)f)dt^^(ij{i,T)))di 



where T > and E'^ ^ are given functions. 

The idea is that J(z) measures the amount of turbulence of the system. So, 
in it is proposed to take ^ bounded and '^(x) = |V x xp, where V x a: is 
the rotational of the three dimensional fields x. In this paper the running cost 
$ satisfies stronger conditions, but we think that our assumptions will be not 
very restrictive. 

Many articles are been devoted to this control problem, and it is proved 
that the optimal control value is a viscosity solution of the Hamilton- Jacobi- 
Bellman(HJB) equation associated to the problem (see, for instance, |4I5II12| and 
references therein). Unfortunately, viscosity solutions are not smooth enough 
to fully justify the dynamic programming approach. 

We follow a strategy proposed in to get smooth solutions. In this 
strategy has been succesfully implemented to study the dynamic programming 
approach for the Burgers equations and for the 2D Navier-Stokes equations. 

After delicate a priori estimates on the Galerkin approximated problem, 
we are able to find a solution for the HJB equation by compactness argument. 
Unfortunately, as for the uncontrolled equation, we are not able to prove unique- 
ness. Moreover, due to the lack of informations on the differentiability of the 
flow with respect to the space variable, we are not able to apply verification the- 
orems (cf. .11 J. Then we shall justify the dynamic programming approach only 
for determinated classes of solution of the controlled equation, which depend by 
a given solution of the HJB equation. 



2 Notations 

Let £) C M'^ be a bounded open set with regular boundary dD in and let 
be the set of the real valued square Lebesgue integrable functions on D. 
We denote by H^{D) the usual Sobolev spaces, and by Hq{D) the space of all 
functions in H^{D) with vanish on the boundary dD. We introduce the Hilbert 
spaces 

H = {x e {L^{D) f : diva; = in £>, a = in dD] , 

V ={xe (i?o(-D))3 : divx in £>} , 

where n denotes the normal unit vector on dD. H (resp. V) is endowed with 
the inner product and norm of (resp. {Hq{D))^) denoted by (■, •) and 

I • I (resp. ((•, •)) and || ■ ||). Moreover, we introduce the unbounded self-adjoint 
operator 

A = PA, D{A) ^ {H\D)f n{H^{D)f nH, 
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where P is the orthogonal projector of (i^(_D))^ onto H and the operator b is 
defined by 

b{x,y) ^ P{{x ■W)y), b{x)^b{x,x) x,yeV. 

is a cylindrical Wiener process defined on a stochastic basis {^1, J-, Tt>o,^) 
with values in H. The operator Q is symmetric, nonnegative, of trace class and 
such that ker Q = {0}. The control z is chosen in the space of adapted processes 

Mr = {ze Ll^in X [0,T]:H),\z\ < R}, 

for a fixed R> 0, and it is subject to a linear operator B G JC.{H) which will be 
specified below. We study our control problem with initial value in D{A). This 
choise will be clearify in the following. 

Thanks to the introduced notations, we can write the problem in the abstract 
form^ 

jdX{t) = {AX{t) + b{X{t)) + Bz{t))dt + Q^^^dW{t), 
\X{Q) ^x,xe D{A). 

Equation H2.1|l is associated to the cost function 

1 



(2.1) 



J{X,z) =E 



($(X(t)) + -|^(t)P)ds + ^(X(T)) 



where T > is fixed and <&, (p : D{A) M are suitable nonnegative functions. 
We stress that the cost function J depends also by the solution of H2.1|l since we 
have no informations about its uniqueness. It is easy to see, by the Ito formula, 
that since the covariance operator Q is of trace class then J{X, z) is well defined. 
We aim at finding an optimal control z* e Mr and an optimal state X* which 
minimize J: 

J{X*,z*) = min { J(X, z) : z !E Mr, X in & solution of (|2.1(l }. 

We follow the dynamic programming approach to solve this problem. Let the 
Hamiltonian F be defined on H by 



F{p) 



^\p\\ ifb|<i?, 
\p\R-^R^, if|p|>i?. 



The Hamilton-Jacobi-Bellman equation associated with our control problem is 

(2.2) 



Dtu = iTr[QM,,] + {Ax + b{x),u,) - F{B*u,) + $(a;). 



[u{0,x) = (p{x), X e D{A), 

where the subscript x means the differential with respect to x. If we are able 
to find a smooth solution u of H2.2|l then the optimal control is given by the 
feedback formula {cf. 7 ) 

z*{t) = ~DpF{B*u^{T - t, X{t))), (2.3) 



^When it will be necessary to emphasize the dependence of X by the initial condition x, 
we write X(t, x) instead of X(t). 
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where the optimal state X is the solution of the closed loop equation 

{dX{t) = {AX{t) + b{X{t)) - DpF{B*u^{T - t, X{t)))dt + Q^/^dW{t), 
\x{0)=x^D{A). 

(2.4) 

Due to the lack of uniqueness of u(t, x) we shall solve the control problem only 
for some classes of solution of H2.1|l . This discussion is detailed in the last 
section. We shall solve equation l|2.2|l under the mild form 



u{t,x)^ Rt(p{x) + I Rt-s{b{-),u,As,-)){x)ds 

JQ 

rt ft 



Rt-sF{B*u,{s,-)){x)ds+ / Rt-sH^)ds, (2.5) 

where Rt is the Ornstein-Uhlembeck semigroup defined by 

Rt^{x) = E[</j(Z(t,a;))], ^ G Cb{D{A)-R) 
and Z{t,x) is the solution of the stochastic equation 

{dZ{t) = AZ{t)dt + Q^/^dW{t), t>0, 
\Z{Q)^x, xcD{A). 

3 Galerkin approximations 

We introduce the usual Galerkin approximations of equation (|2.1(l . For m G N, 
we define by Pm the projector of H onto the space spanned by the first m 
eigenvectors of A. Then we set, for x ^ H, bm{x) = Pjnb{Pmx) , Qm = PmQPm 
and Bra = PmBPm- We consider the control problem in finite dimension for 
the approximated equation 

('dX"(<) = (AX"(<) + 6™(X"(t)) + BraZ{t))dt + Q]l^dW{t), 

\x^{Q)=PmX,xeD{A), 
which consists in minimizing the cost function 



J m (^m ) 
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(<i>(X™(t)) + -\PraZ{t)\^)ds + ^(X'"(T)) 



where z G Mr- ■ Then, we consider the approximated equation of l|2.5|l . i.e. 

U^{t,x) = RTvi^) + f RT-s{brn{-),U^{sr)){x)ds 



RT .F 



•{B*^u'^{s,-)){x)ds+ f RT_M^)ds, (3.2) 

JO 



where {i?™}t>o is the Galerkin approximation of the Ornstein-Uhlembeck semi- 
group. In order to approximate H2.5(l by 1)3. 2|l . we need some a priori bounds 
for u"^{t,x). So, we consider the Feynman-Kac semigroup {Sp}t>o defined by 



STfix) = E[e--^^* \'^^'"^'-'''^\^'^'ip{Y"'{t,x))\ , 



(3.3) 
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where Ym{t,x) is the sohition of the finite dimensional stochastic equation 

¥""{0) ^ Prnx, xeD{A). 
(|3.2() has an unique solution x), which is also the solution of 

u'-it,x)^Sr^{x) + f Sr.MA-\'u"\s,-)){x)ds 
Jo 

Sr^,FiB*^u^is,-mx)ds+ [ Sr-sH^)ds, (3.4) 

Jo 

and of 

r Dtu'''{t,x) = iTr[Q™M™(t,x)] + (Ax + x)) 

I -F(i?>^(t,x)) + $(a;), (3.5) 

[u"^{0,x)^ip{x), 

where {t,x) € [0,T] x PmH . Thanks to this property, we shall be able to find 
the a priori bounds that we need. By a classical computation based on the Ito 
formula (cf. (7,) we find that the optimal control zj^ is obtained by taking 

z:^{t) = -Dj,F{Blul\T - t, X"(i))), 

where X'"(t, x) is the solution of the closed loop equation 

r dX"'{t) - {AX"\t) + bmiX'^'it))^ 

< ^ -DpF{B*^u'^{T -t,X'^{t))))dt + Qll''dW{t), (3.6) 

{x"'{0) = P^x,x£D{A). 

Since (|3.6|1 is in finite dimension, it is easy to prove existence and uniqueness of 
a solution. 

We recall that a detailed survey on HJB equations on finite dimensional 
spaces may be found in |10| . The infinite dimensional problem has been recently 
developed, see [Zj. 

4 Functional spaces 

Let {E, I • \e) be a Banach space. If </? : D{A) E and h <E H wc set 

s^O s 

when the limit in E exists. The space D(A) is endowed with the graph norm. 
We define the following functional spaces: 

• Cb{D{A); E) is the space of all continuous and bounded functions from 
D{A) to E, endowed with the norm 

\(p\o ■■= sup \'pix)\E, e CtiD{A);E). 

xeD{A) 
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• For any fc e N, CkiD{A); E) is the space of all continuous functions from 
D{A) to E such that 

mk,A sup , I ■ < oo. 

• For any A: G N, (£>(/!); is the space of all functions of Ck{D{A)\E) 
such that 

\^Was ■■= sup — ^ I . — < oo. 

• For any /c e N, ffc(Z3(^); is the space of aU function e Ck{D{A)\E) 
such that 



\A{x-y)\{\ + \Ax\ + \Av\Y 



= sup 1,,^ ^,,1,, , 1,^1 , < OO- 



• For any /c £ N, a G (0, 1) the set ^_^-^^{D{A); E) denotes the space of 
all functions of Cl{D{A); E) such that 

I I \{-A)-"D^{x)\e , 

^S fc-C-A)".! := sup , I . < oo. 

xeD{A) (1 + \Ax\)'' 

For any (S > we denote by D{{—A)^^) the dual space ofD{{-AY). We shall 
identify the space H with its dual H* . Hence, the embeddings D{{—AY) C H 
C ^((-A)-*) hold. 

5 Hypothesis on the operators Q, B 

Following P|[S] we assume that 

Tr[{-AY+9Q] < +00 (5.1) 

for some 5 > and that 

\Q-^'^x\<Cr\{~AYxl -ix^D{{-AY), (5.2) 
for some r E (1, 3/2) and Cr > 0. Then, we shall denote by e the quantity 

Notice that by the hypothesis on r we have e £ (0, 1/2). If Z{t) denotes the 
stochastic convolution with covariance operator Q, i.e. Z{t) is the solution of 
the linear stochastic equation in H 

Z{t) = f e^^-'^'^y^dWis), t > 0, 



then hypothesis (|5.1|l implies that 

E[|(-.4)i+9/2z(t)|2] = Tr[(-A)i+9Q] < 
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Since Z{t) is gaussian, by the factorization method (see, for instance, ^) it 
foUows that for any p > it holds 



sup \{-Ay+3/^Z{t)\P 
te[o,T] 



< 



Consequently, setting 



{t~s)A 



it is clear that 
E 



Zm.it) = / e 



sup \{-Ay+s/^Zm{t)\P 

*6[0,T] 



c(p)Tr[(-A)H 



dW{s) 



On the linear operator B we suppose that 

B:H — > D{{~A)-i) 

for some 7 > 1 — e. This implies that for some c > and for all z E D{{—A)~'^) 
it holds 

\B*z\<c\{-A)-''z\. (5.4) 



6 A priori estimates I 

In this section we prove some useful estimates on the Feynman-Kac semigroup 
We omit the proof of well-known results. We use the following estimates 
on the bilinear operator b{x,y) (see, for instance, pill4p. 

Lemma 6.1. There exists c > such that for all x,y € D{A), z £ H. 

{b{x,y),{-Ay/'z)<c\Ax\\Ay\\z\. 
Detailed proof of the following lemmata may be found in "S". 

Lemma 6.2. Let k e N, ip e Ck{D{A);M.) n Ck,(^Ay/2{D{A);M.) and a € 
(1/2, 1). Then there exists c > such that if K is sufficiently large we have 

l'5rVlfc,(-A)V2,i < ct-^/^i\ip\k^A + \^\k,A,l), t > 0, 
\Sr^\k,i~A)^,l < cf-W^lk^A + \^\k,A,l) t > 0. 

Proof. The first estimate is proved in Lemma 4.4 in The second one follows 
by interpolation between the first one and the estimate 



\Sr\k,A,l < c{\(p\k,A + \v\k,A,l), 

which is proved in Lemma 4.2 of 



□ 



Lemma 6.3. Let k e N, ip e CkiD{A);R), a e (1/2,1). Then there exists 
c > such that if K is sufficiently large we have 

\ST^\k,i-A)^A < Cl(l + t^+'-^Mk,A, t > 0, 
where e is defined by (15. 
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Proof. By Lemma 4.1 of 3 , we know that there exists c > such that 

\Sr^\k,A,i < c(l + t'-^Mk,A, t > 0, m e N. 



Hence, taking into accomit Lemma |t) . 21 and the semigroup property of S*™, we 
have 

= |5't72'5't72</'lfc,(-A)"4 < C(l + (t/2)""^)(|S'™2¥'|fc.A + \S^2^\k,A,-i) 

< c(l + (t/2)"-i)(|^|,,^ + (1 + {t/2r-^Mk,A) 

which imphes the result. □ 

Lemma 6.4. LetkeN, (pe Cl{D{A):R), a e (1/2,1) and a G (3/4,1). Then 
there exists c > such that for all m Cz N, t > it holds 

\{-Ar"D'sr^{-)i-A)--\k,A < c(i + t^+"+'-')i\ip\k,A + i^kAA). 

Proof. The resuh follows by Lemma 4.5 and Lemma 4.8 in and by arguing 
as in the previous Lemma. □ 

7 A priori estimates II 

In this section, we assume that ip, <i>, a satisfy the following conditions 
ip{x), $(x) > xe D{A); 

<p, <^ e Cb{D{A);R) n £2{D{A);R) (7.1) 

a e (1 — e, 1), a < 7 

where p £ N is fixed, c > and e, 7 are defined in H5.3|l . I|5.4|l respectively. We 
are going to establish some estimates on u™'{t,x), Du™'{t,x), D^u™'{t, x). 

Proposition 7.1. There exists c > such that for all m it holds 

sup |u™(t,-)|o < c{\ip\a + |$|o) 
te[a,T] 

Proof. Since m™ is the function associated with the approximated control prob- 
lem we deduce that 

< u™ (T, x) = min J„, (z) < J,„ (0) 

zeAii?nL^(nx[o,T];P„H) 



-cE 



T 

^{X''\s,x))ds^>p{X"\T,x)) 







<T|$|o + Iv'lo- 

We could consider the same control problem in [0, t] and obtain 

0<u™(t,x) < |(^|o+r|$|o 
This completes the proof. □ 
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Proposition 7.2. There exists c > such that for aZ/ to G N it holds 

\u"\t, ■)\2,i-A)^,i < cr-i(i^io + \ip\s., + mo + ms,). 

Proof. Let us assume that t/?, $ G CbiD{A);R) n C^{D{A);R). Taking into 
account H3.4|l we have 

•)|2,(-A)-,l < II + /2 + /3 + /4 

where 



Il = 


i5r'p(-)i2,(-A)=,i> 






f\Sl'U\A-\'u"^is,-))\u-Ay 
Jo 




h = 


f \ST^,F{Blu^{sr))\2,i^A)^ 
Jo 




/4 = 


1 \ST_M-)\2,(~Ar,ids. 

Jo 





Ii and /4 are estimated as in Lemma 16.21 For I2 we have that 

11^ • \'u"\s, ■)\2.A < C|U"(5, ■)\o < ci\ip\o + |$|o) 

by Proposition 17. II Hence, taking into account Lemma [6. 31 and that a + e > 1, 
it holds 

h<c [\l + {t- sr+'-^)\\A ■ ■)\2.Ads < cd^lo + |<i>|o). 

Jo 

For /a we have that 

\SrF{B*^uT{s, •))|2,(-A)=,i < c(l + .))|2,A. 
by Lemma [6.31 Since a < 7, by 15.4(1 it follows 

\F{B*^u^{s,-))kA<c\{~A)-''u^{s,-)\2^A 

< C|(-A)-"<(S, .)|2,A = C|li"(s, ■)\2.i-A)^,l, 

and so we find 

h < C f\l + it- sr+^-')\u"^{s, ■)\2,(-A)^,lds. 

Jo 

Finally, by gathering all the estimates on /i, I2, I3, Ii we find that for some 
c > it holds 

-hx-A)^.! < Ct"-^(|^|0 + |^|2,A,1 + |<i>|0 + |$|2,A,l) 

+C / (l + (t-s)"+--2)|„"(s,.)|2,(-A)".lrfs. 
JO 

Then, by Gronwall's lemma (see, for instance, iX^), we have 

^"'{t, ■)\2.X~A)^.l < Ct"~\\v\0 + |^|2,A,1 + |$|o + m2,AA) 

since a + e > 1. Now notice that all the estimates above are done in the 
finite dimensional space PmH. Hence, if $ G Cb{D{A);R) r]£2{D{A);M.), we 
obtain the result by approximating uniformly if, <i> by functions in Cb{D(A); R) 
nCi(D(A);]R). □ 
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The following two results will be proved with a similar argument, by using 
Lemma 16.41 

Proposition 7.3. Let a G (3/4, 1) such that a + a + e > 2, where a and e 
are defined in (|7.1|l . H5.3|l respectively. Then there exists c > such that for all 
m it holds 

|(-A)-"W"(t,-)(-A)-n4,A 

< cr+"+^-3(|^|o + l^lf, + 1$ lo + 1$ lej. 

Proof. Notice that by the approximation argument described in ProDOsition l7.2l 
it is sufficient to prove the claim for (/J, $ £ CbiD{A);R) n C^{D{A);R). By 
(|3.4|) we write 



where 



ji = i(-A)-"D25rv(-)(-^)-n4,A, 

J2 = I \{-A)-^D^Sr^,{\A-\^u'^{s,-)){-A)-^W,Ads, 

JQ 

J3 = /* \{~A)-'^D^SllMBL<isr))i-A)-^UM^, 
Jo 

Ja^ I \{-A)-^D^SYLM-){-A)-^U,Ads- 

Jo 

Jl and J4 are estimated by Lemma 16.41 For J2 we have, by Lemma 16.41 

J2< c f\t-sr+"+'-m\A-\^u"'is,-)U,A + \\A-\^u"^sr)kA.i)ds 
Jo 

<c [ {t~ sr+"+'-''{\u-\s, ■)\2,A + Ols.A + \u^{s, ■)\2.A.i)ds. 

Jo 

Consequently, taking into account Proposition lT.ll and Proposition l7.2l it follows 

J2 < c{\^\o + \^\2,AA + |$|0 + |*|2,A,l) [ {t~ sr+'^+'-^'s'^-^ds 

Jo 

< c(|^|o + \^\2^AA + |$|0 + |$|2.A.l))^'"+"+'"'- 

For J3 we have, by Lemma 16.41 

J3 < c [\t - sr+'^+^-H\F{B*,u^{s, ■))U^A + \F{B*^u^{s, ■))U,A,i)ds. 
Jo 

Hence, since \F{B*^ipx{-))\ < c\B*ipx{-)\, for a defined as in H7.1(l we have that 
J3 is bounded by 

c fit - •)|4,(-A)^,i + \{-A)-^D'u^--{t, ■)(~A)-\^A)ds. 
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Taking into account Proposition Proposition 17 . 31 we find 

^3 < c(|^|o + W\2,AS + l*lo + l*|2,A.l)) 


So, the result follows by gathering the estimates on Ji, J2, J3, J4 and by applying 
Gronwall's lemma. □ 

Proposition 7.4. There exists c > such that for all m G N and t > it holds 
Proof. By we write OU.i-A)!/^,!! < /i + ^2 + ^3 + -?^4, where 

Ki = |5'"V(-)l4,(-A)i/2,i, 

if2= /Vr.(l^-|'«(s,-))l4,(-A)V2,ids, 

Jo 
Jo 

K,= f |5™,$(-)l4,(-A)V2,id.S. 

Jo 

Ki, K2 are estimated by Lemma |^21 For K2 we have by Lemma [6. 21 

K2 < C fit - . .))|4,A + ||A • ■))U,A.l)ds 











Then, taking into account Proposition 17. 31 and Proposition 17.21 it follows 

K2<c{\^\0 + \V^\2.AA + m0 + m2.AA) f [t - s)-^'^ S^^-^S 

Jo 

< c{\ip\o + \f\2,AA + l^-lo + l*|2,A,l) 

since a > 1/2. For K3 we have 

K3 < C f\t - s)-^/'{\F{B:^u:\t, .))|4,A + I W„<(S, ■))U,AA)ds. 

Jo 

by Lemma 16.31 Hence, for a defined as in (|7.1|l we have 

K3 < c f\t - sr'^^\u"\t, ■)\4,(-A)^,i + \{-A)-''D'u"\t, ■){-Ar'U.A)ds. 
Jo 

Consequently, taking into account Proposition l7.2l Proposition 17. 31 we find 

K3 < c(|^|o + \(p\2,AA + |*|o + |$|2,A,l) 
^0 

< Cd^lo + |^|2,A1 + |*|0 + |$|2,A,l)(l + t"+-+--5/2). 



Now the conclusion follows by gathering all the estimates on Ki, K2, K3, K4 
and by taking into account that a + a + e — 5/2 > —1/2. □ 
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The proof of the following proposition is the same of Proposition 3.6 in 8 ; in 
addition we have the hamiltonian term, but it is treated by the same arguments 
of Proposition (|73, lESJ, (fTIIl . 

Proposition 7.5. For all <, s > 0, m e N and x G D{A) we have: 

W'\t,x) - < cd^lo + W\e, + |<i>|o + |<i>|£j(l + I^^D^ 

X {\t - s\a'^ + \t- .s|i/2 + \A{e'^ - e'^)x\). 

8 Construction of a solution 

In order to prove existence of solutions of 12.5|l we proceed by a compactness 
method. Let (5 > and set Ks = {x ^ D{A) : \Ax\ < S}. 

Theorem 8.1. Let us assume that H5.1|l . H5.2|l . H5.4|l . H7.1(l holcP. Then there 
exists a subsequence {u"^"{t,x)} of {u"^{t, x)} and a continuous function u : 
[0, T] X D{A) —> M such that for all to G (0, T), S > the following statements 
are satisfied: 

(i) u{t, •) e Cb{D{A);R) for all t G [0,T]. Moreover, 

lim u""(t,a;) = (8.1) 

rrin — ^oo 

uniformly on [0,T] x Kg; 
(u) u{t, ■) e C^{D{A);R) for all t e (0,T]. Moreover, 

lim \{~A)-"(Du'''"{t,x)-Du{t,x))\^0, (8.2) 

rUn — >oo 

uniformly on [to,T] x Kg; 

(iii) for any (t, x) G (0, T] xD(A) there exists the directional derivative Dhu{t, x) :— 
Du{t,x) ■ h in any direction h G D{{—Ay/^) and 

lim Dm™" {t, x) ■ h = Du{t, x) ■ h, 

uniformly on [to,T] x Kg. Moreover, there exists c > such that for any 
t G (0, T] it holds 

\Du{t,x) ■ h\ < ct-^'^{l + \Ax\f\{-AY'''h\- 

(iv) u(t,x) is a mild solution of the Hamilton-Jacobi-Bellman equation (|2.2(l . 

Proof Let y^, $ G Cb{D{A);R) D £2{D{A);E.). We deduce from Propositions 
OOthat for any S > 0, to e (0,T), a G (3/4,1) there exists C{6,to,'^,g) 
such that Vm G N, t, s > io it holds 

\u"\t,x)-u^-is,y)\<C iS,to, ^, $)(|t - + |(-A)i/2(:, _ y)\) 

^Existence of solutions for equation 12. 2i may be proved by assuming that for some p > 
the functions (p, <& satisfy (p{x), ^(x) < c(l + Ixl^"), x & H , p > 0. However, since we shall 
need that in the control problem ip, # are bounded, we have omitted this case. 
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and 



I (- A)-" {Du"^ {t, x) - Du^it, y)) | < C(<5, to, V', $) I (- A)^ (x - | . (8.3) 

Then, by the Ascoh-Arzela Theorem and by a diagonal extraction argument, 
it follows that there exists a continuous function u : (0,r] x D{A) — > M such 
that u{t,-) e Cb(i:'(A);M) n CliDiA);^) for any t e (0,T] and a subsequence 
{mn}n£n of N such that 1)8.1(1 . 1(8. 2|l holds, uniformly in [to, 2^] x -f'^s, for any 
to G (0,T]. Hence (ii) follows. 

Let us set u{Q,x) = ^{x). In order to prove that u : [0, T] x D{A) R 
is continuous, it is sufficient to check that 1(8.111 holds uniformly in [0, T] x Kg- 
Before proving this, we prove (iii). Let m, n £ N, /i G A)-^/^) and h' G 

D{A). Then for any t G (0, T] we have 

\{DvJ^{t,x) - DvJ'it.x)) ■ h\ 

< \Du"^{t,x) ■ {h-h')\ + \{Du"'{t,x)~Du''{t,x))-h'\ + \DvJ'{t,x) ■ {h-h')\ 

< t-i/2c(<^)(l + |At|)4|(-A)1/2(/i - h')\ 

+ \{-A)-''{Du"'it,x) ~ Du"{t,x))\\{~A)"h'\, 

by ProDOsition l7.4l and ((8.3(1 . Hence, since A)^/^) is dense in D{A), the se- 
quence {Du™"^ {t, x) ■ h}m£N is Cauchy in M, uniformly in [to, T] x Ks. We denote 
the limit by Du{t, x) ■ h: necessarily it coincides with the Gateaux derivative 
Dhu{t, x) along the direction h. Then (iii) is proved. 

Now we prove (iv). To do this, we shall check that the right-hand side of 
(13.2(1 converges to the right-hand side of ((2.5(1 . By Proposition ((7.4(1 we have 

\{~A)-'^'Du"\t,x)\i,A, \{-A)-'^'Du{t,x)U,A < c{ip,g)t-'/\ 
Then by Lemma Ifi.ll and (iii) we find 

(6,„(Z'"(t - s, x)), Du'^is, Z'"(t - s, x))) < c{ip, g)t-^^\l + |ylZ™(t - s, x)\ f. 
Similarly, by ((5.4(1 and by Proposition 17. 21 we find 

F{B*^u:^s, Z^{t - s,x))) < s-"c{ip,g)il + \AZ"^{t ~ s,x)\f. 

Then all the integrals in ((2.5(1 . ((3.21) are well defined, and by letting to„ — > oo 
in ((3.2(1 . Statement (iv) follows by the dominated convergence theorem and 
by the well known properties of the Galerkin approximations of the Ornstein- 
Uhlcmbeck semigroup. To complete the proof of (i), it is sufRcient to notice 
that the convergence is uniform in [0,T] x Ks- □ 

8.1 Martingale solutions of the controlled equation and 
main result 

The goal of this section is to prove that there exists a solution, in a suitable 
sense, of equation ((2.1(1 . We shall prove that the process X{t, x) is a solution of 
equation ((2.1(1 in the following sense (cf 0): 

Definition 8.2. We say there exists a martingale solution of equation ((2.1() if 
there exists a stochastic basis {il, T, {^t}telo,T] > P), a cylindrical Wiener process 
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W on the space H and a progressively measurable process X{t, x) : [0, T] x ^ 
H with P-a.s. paths 

x{; x){uj) e C([o, T]-D{{-A)-^))) n i°°([o, T]■H)r^ t]-v) 

such that the identity 

{X{t,x),y)+ I {AX{s,x) + b{X{s,x)) + Bz{s),y)ds 

= ix,y)+ f\Q'/''dWit),y) 
Jo 

holds true for all te[0,T],y£ D{A). 

By Ito's formula it follows easily (see, for instance, [H]) 

Lemma 8.3. There exists c > such that for any x G H , m G N, z Cz A4r the 
following estimate holds: 



E 



sup \X„,{t)\'+ / \\Xrn{sWds 
Q<t<T 



T 

2, 



<c(l + |xp+Tr[g]), (8.4) 



It is well known (see, for instance, 9 ), that l|8.4|l implies that the family of 
laws {£(X™(.,x))}^6N is tight in L^^([6,T]; Di{-Ar)) n C([0, T]; D((-^)-i) 
for any a < 1/2. Then, for a fixed cr < 1/2, there exists a probability law i^x 
on L2^([0,T];L>((-A)'^)) n C([0, T]; i:>((-A)-i)) and a subsequence X""'{-,x) 
such that C{X™''' {■ , x)) — > i^a, weakly, i.e 



hm / ^(0/:(x™H-,^))(rfO 

mfc-*oo JL2^,([0,T];D((-A)-))nC([0,T];_D((-A)-i)) 

V'(o^x(rfe), (8.5) 

L2^^([0,T];r>((-A)-))nC([0,T];n((-A)-i)) 

for all bounded and continuous ^ : Ll^dO, T];D{{-Ay)) n C([0, T]; D{{-A)-^)) 
— > R. Moreover, by the Skorohod theorem (see, for instance, (HI), there exists a 
probability space {Qx, ^xi'^x) and a stochastic process X{-,x) with law such 
that X'"'=(t,a;) ^ X{t,x) P^-a.s.. 
So, we have proved the following 

Theorem 8.4. For any z G M.R and for any x G D{A) there exists a probabil- 
ity space (fia;, .Fa;, Px), a process X{-^x) and a sequence of stochastic processes 
{X"^" x)}k£Ti defined on {fix, J-'x,^x) such that 

(i) The processes X"'"(-,a;) are solutions of the Galerkin approximated equa- 
tions ()3.1|l ." 

(ii) The sequence X™"(-,x) converges Pj^-a.s. to the process X{-,x); 

(iii) X{-,x) is a martingale solution of (|2.1(l . 

Let us assume that there exists a solution u{t, x) of 1)2.511 such that for a sub- 
sequence u™"(t,a;) of solutions of (|3.2I) the statements (i)-(iv) of Theorem 18.41 
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are satisfied and let us denote by X"^"{-, x) the solution of the TO„-Galerkin ap- 
proximated equation (|3.6() . Since PmZ* G Mr we can argue as in Theorem (|8.4|) 
to find a subsequence {m^}„gN of {mn}neN and a probability space {flx^J-xi^x) 
such that the solutions X"^^{-,x) converges Pa;-a.s. to a process X{-,x). In or- 
der to prove that X{-,x) is a martingale solution of the closed loop equation 
H2.4(l we have to define the optimal control z* as in H2.3|l . Then, since u{t,x) 
is defined for x G D{A), we need that the martingale solution X{t,x) of ()2.1|l 
belongs to D{A). To do this, we have the next 

Lemma 8.5. For any 5 G (l/2,min{l + g,l + 27}] there exists a constant 
c{d) > such that for any x £ H , m G N and t G [0, T] it holds 



E 



where 

Proof. We apply the Ito formula to 

Fsix) - 

We obtain 



|(-^)(i+^)/2x"^(g^a^)|2 

(l + |(-A)V2X'"(s,x)|2)e. 

2(5 + 1 



ds 



<c{S) 



2(5-1 



1 



(1-H |(-^)'5/2a;|2)fl.-l- 

r |(_^)(l+^)/2X"(s,x)|2 



2(05 


- 1)E 


+ {0s 


- 1)E 


+ {0s 


- 1)E 



{l + \i-Ay/^X--{s,x)\^) 
{X^{s,x)),{~AYX^is,x)) 



-ds 



(l + |(-A)'5/2X'"(s,a;)|2)e<5 
^ ((-A)*X™(s,a;),BzW) 



-ds 



(1-|-|(-^)V2X™(S,X)|^ 



1 



5 - 1)E 



(l + |(-A)V2Xm(s,a;)|2, - 

^ |Q,V'(-A)*/2X'"(s,a;)p 



-E 



(l + |(-A)V2Xm(s,x)|2) 



1 



5 + 1 



tt[qU~aY] 

ds 



By Lemma l6. II and by interpolation we find 

< c|(-A)*'/2;f-|2||±i ^ 1 |(_^)(l+5)/2;^m|2^ 

Moreover, since S < I + 2"f and z is bounded we have 

{{-A)^X"\Bz)<c\B*{-A)^X"'\ < c|(-A)*-^X'"| 

<c|(-A)(l+'^')/2X™| < C-l- i|(-A)(l+'')/2j^m|2^ 
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By (inm we know that Q]L^{~Afl'^ is a bounded operator. It follows that the 
last three terms in the right-hand side are bounded. We deduce that 

|(_^)(i+5)/2x"(s,a;)|2 



E 



< E 



(H-|(-A)V2X"(s,x)|2)e^ 
|(_A)5/2x™(s,a;)|2^ 



ds 



-ds 



ciS). 



Finally, since 9-^ — 



□ 



(1 + |(-^)V2X™(S,I)|2)«. 

J, the result follows. 

Arguing as in Lemma 7.5 of ^ it follows the next 

Lemma 8.6. Under the assumptions of Theorem 18.41) the sequence of processes 
{X"^"{-,x)}neN, X{-,x) also satisfies 

lim X"'"{t,x) = X{t,x) in D{A), dt xP,^-a.s in[0,T] xQ^. (8.6) 



Main result 

For any sequence {r7i„}„gN C N and any z G M-r we denote by MS{{mn},z) 
the set of all martingale solutions X{-,x) of H2.1|l such that for some subsequence 
{m^}„6N C {m„}„gN we have 

X{-,x)^ lim X""(-,a;) inlaw, 

where X"^^{-,x) is the solution of the the m^-Galerkin approximated equation 
We have the next 

Theorem 8.7. Let us assume that the functions u{t, x), {u™"(i, a;)}„gN satisfy 
statements (i)-(iv) of Theorem \ H.1\ Then, for any x G D{A) there exists a 
martingale solution X{-,x) of the closed loop equation (|2.4(l . Moreover, the 
control z* defined by 

z* = ^DpF{B*u,{T - t, X{t, x))) 

verifies 

u{T,x) = J{X{-,x), z*) — min J{X{-,x),z). 

zeMa 
X(-,x)eMS({m,^},z) 

Proof. Arguing as in Theorem 18.41 there exists a subsequence {m^}„gN of 
{"T-nlneN; a probability space {Vlx, J'x^'^x) and a sequence of processes {X™" (•, a;)}„gN, 
X{-,x) defined on Vl^ which satisfies statements (i), (ii) of Theorem 18.41 and 
(18.51) . Moreover, by Lemma [8. 61 we can assume that (|8.6|l holds. Consequently, 
by statement (ii) of Theorem l8.1l it is easy to see that 

lim B*^, u7'"{T -t,X'<{t,x)) ^ B*Ux{T -t,X{t,x)), dtx P^-a.s.. 

Hence, since \DpF\ < R, we can apply the dominated convergence theorem in 
order to have 



lim 

m' — >oo 



{D.pF{B:^, u^-{T-s,X^^^{s,x))),y)ds 







{DpF{B*u{T - 3,X{s,x))),y)ds 



(8.7) 
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a.s., for all t E [0,T] and all y G D{{—A) ^). Now, arguing as in Theorem 
it follows that X{-,x) is a martingale solution of (|2.4|l . 
For the second statement we notice that it holds 

u(T, x) - [ifiiXiT, x))] = lim (u'< (T, x) - [(^(X< (T, x))] 

Since x) is solution of H3.5|) . by a standard computation based on the Ito 

formula (cf j7j) we have, for any m e N, 



m'"(T,x) 



(<i>(x"(.,x)) + -|z;;(.)|2)d. + ^(x'"(r,x)) 



Then, taking into account H8.7|l and that Lp, $ are bounded, we can apply the 
dominated convergence Theorem to yield 

u{T,x) = J{z*){x). 

Now notice that for any n G N the process is the optimal state for 

the m^-Galerkin approximated control problem H3.1|l . which implies 



for all z G Mr,- Then, by letting —^ oo, we conclude the proof. 



□ 
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